13,hICCRTS:  C2  for  Complex  Endeavors 


“C2  Network  Analysis:  Insights  into  Coordination  &  Understanding” 

Topic  6:  C2  Assessment  Tools  &  Metrics;  Topic  2:  Networks 
Author:  Jeffrey  T.  Hansberger,  Ph.D.,  Craig  Schreiber,  Ph.D.,  &  Randall  D.  Spain,  M.S. 

POC:  Jeffrey  T.  Hansberger 
Army  Research  Laboratory 
115  Lakeview  Parkway,  Suffolk,  VA  23435 
757-203-3431 
ihansberger@arl.army.mil 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JUN  2008 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2008  to  00-00-2008 

4.  TITLE  AND  SUBTITLE 

C2  Network  Analysis:  Insights  into  Coordination  &  Understanding 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Army  Research  Labor  atory,l  15  Lake  view  Parkway,Suffolk,VA,23435 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

13th  International  Command  and  Control  Research  and  Technology  Symposia  (ICCRTS  2008),  17-19  Jun  2008,  Seattle,  WA 


14.  ABSTRACT 

The  distributed  cognitive  framework  (Hutchins,  1995)  provides  a  structured  and  theoretical  approach  for  analyzing  cognitive 
characteristics  beyond  that  of  a  single  individual  to  that  of  a  system  comprising  of  multiple  individuals,  tools,  and  the  task 
environment.  Among  some  of  the  attributes  of  a  distributed  cognitive  system  are:  1)  coordination  across  agents  2)  mental 
models,  3)  situation  assessment,  4)  memory  demands,  5)  adaptability,  and  6)  workload  management.  This  paper  will  address 
recent  efforts,  tools,  and  approaches  on  measuring  and  analyzing  two  of  these  distributed  cognitive  attributes  through  network 
analysis,  coordination  across  agents  and  mental  models.  Network  analysis  was  applied  with  different  methods  and  emphasis  to 
both  attribute  areas.  The  analysis  of  the  coordination  across  agents  applied  network  analysis  to  analyze  the  patterns  of 
interactions  across  human  and  technological  agents  over-time.  Collecting  data  related  to  coordination  over  time  required 
specific  capabilities  that  was  not  readily  found  among  observational  data  collection  tools  and  therefore  required  a  custom 
program  that  we  designed.  Description  of  the  requirements  and  implementation  of  this  new  observational  network  analysis  tool 
as  well  as  methods  to  visualize  longitudinal  network  change  is  addressed.  The  analysis  of  mental  models  also  utilizes  a  basic 
network  analysis  approach,  namely  structural  knowledge.  The  examination  of  structural  knowledge  to  assess  individual  mental 
models  will  be  discussed  to  provide  insight  into  understanding.  Specifically  applied  to  C2,  this  analysis  can  provide  insight  into 
the  commander  and/or  staff?s  understanding. 


15.  SUBJECT  TERMS 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF  PAGES 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Same  as 
Report  (SAR) 

33 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


13th  ICCRTS:  C2  for  Complex  Endeavors 


C2  Network  Analysis:  Insights  into  Coordination  &  Understanding 


Abstract 

The  distributed  cognitive  framework  (Hutchins,  1995)  provides  a  structured  and 
theoretical  approach  for  analyzing  cognitive  characteristics  beyond  that  of  a  single 
individual  to  that  of  a  system  comprising  of  multiple  individuals,  tools,  and  the  task 
environment.  Among  some  of  the  attributes  of  a  distributed  cognitive  system  are:  1) 
coordination  across  agents  2)  mental  models,  3)  situation  assessment,  4)  memory 
demands,  5)  adaptability,  and  6)  workload  management.  This  paper  will  address  recent 
efforts,  tools,  and  approaches  on  measuring  and  analyzing  two  of  these  distributed 
cognitive  attributes  through  network  analysis,  coordination  across  agents  and  mental 
models.  Network  analysis  was  applied  with  different  methods  and  emphasis  to  both 
attribute  areas.  The  analysis  of  the  coordination  across  agents  applied  network  analysis  to 
analyze  the  patterns  of  interactions  across  human  and  technological  agents  over-time. 
Collecting  data  related  to  coordination  over  time  required  specific  capabilities  that  was 
not  readily  found  among  observational  data  collection  tools  and  therefore  required  a 
custom  program  that  we  designed.  Description  of  the  requirements  and  implementation  of 
this  new  observational  network  analysis  tool  as  well  as  methods  to  visualize  longitudinal 
network  change  is  addressed.  The  analysis  of  mental  models  also  utilizes  a  basic  network 
analysis  approach,  namely  structural  knowledge.  The  examination  of  structural 
knowledge  to  assess  individual  mental  models  will  be  discussed  to  provide  insight  into 
understanding.  Specifically  applied  to  C2,  this  analysis  can  provide  insight  into  the 
commander  and/or  staffs  understanding. 

Keywords:  Social  network  analysis,  pathfinder,  structural  knowledge,  visualization, 
coordination,  collaboration,  distributed  cognition,  mental  model,  command  and  control, 
longitudinal  analysis 


Introduction 

Command  and  Control  (C2)  is  a  complex,  dynamic,  and  often  vaguely  defined  area  in 
military  operations  and  actions.  It  is  known  by  many  different  names  such  as  battle 
command  and  command,  control,  communications,  and  intelligence  (C4I)  in  an  attempt 
to  further  define  it  or  better  describe  the  primary  components  of  C2  (Foster,  1988). 
Crumley  and  Sherman  (1990)  reviewed  a  large  body  of  C2  work  and  described,  “most 
[C2]  theorizing  as  simplistic  rather  than  autistic,  and  to  note  that  the  major  problem  in  the 
field  is  not  that  it  is  ‘convoluted  and  idiosyncratic’  but  that  too  much  of  it  lacks  a  clear 
focus”  (p.  viii).  While  not  suggesting  a  unifying  theory  for  C2,  this  paper  suggests  that 
C2  and  all  of  its  working  parts  and  components  display  attributes  other  studied 
distributed  systems  share  (e.g.,  Hutchins,  1995).  By  systematically  applying  such  a 
theoretical  framework  for  analysis  and  assessment  of  C2  systems,  we  hope  to  illuminate 
before  unseen  patterns  and  facilitate  enduring  insight  and  understanding. 
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Distributed  cognition  (Hutchins,  1995)  is  a  theoretical  framework  that  emphasizes  the 
distributed  nature  of  cognitive  phenomena  that  goes  beyond  the  cognitions  of  a  single 
individual.  This  approach  focuses  on  the  functional  system  as  a  whole  to  examine  the 
relation  between  individuals,  the  task  environment,  and  artifacts  (tools  &  technologies) 
used  for  task  completion.  This  approach  has  been  applied  to  several  domains  in  the  past 
such  as  naval  navigation  and  the  aviation  domain  and  can  provide  a  more  comprehensive 
understanding  of  the  functional  system  including  the  interactions  of  system  components. 

Within  such  a  system  perspective,  there  are  several  cognitive  attributes  (Woods, 
Johannesen,  Cook,  &  Sarter,  1994)  that  can  be  affected  by  the  other  elements  within  the 
system.  Many  of  these  attributes  have  direct  ties  to  major  Command  and  Control  (C2) 
functions  and  requirements  (Hansberger,  in  press). 

Distributed  Cognitive  System  Attributes: 

1)  Coordination  across  agents 

2)  Mental  models 

3)  Situation  assessment 

4)  Memory  demands 

5)  Adaptability 

6)  Workload  management 

Each  of  these  component’s  importance  and  relevance  can  vary  according  to  the  situation 
and  task  environment.  This  paper  will  focus  on  two  of  these  attributes,  the  coordination 
across  agents  and  mental  models  in  order  to  provide  some  detail  in  their  measurement 
and  analysis  related  to  C2. 

Coordination  across  agents  refers  to  the  interaction  and  communication  between  human 
agents  along  with  human-to-computer/automated  agents.  The  consideration  of  human- to- 
computer  agents  is  important,  as  it  is  the  element  that  broadens  social  network  analysis 
beyond  the  person-to-person  interactions.  Our  efforts  in  collecting  and  visualizing  this 
type  of  data  over  time  will  be  addressed  in  the  first  section  of  “Coordination  across 
Agents”.  The  mental  model  attribute,  on  the  other  hand,  focuses  on  the  structural 
knowledge  and  understanding  an  individual  has  on  a  topic  or  domain.  The  structural 
knowledge  consists  of  both  relevant  domain  concepts  as  well  as  the  relationship  among 
those  domain  concepts  with  one  another.  The  approach  and  methods  used  to  measure  and 
collect  data  related  to  mental  models  and  structural  knowledge  is  addressed  in  the  second 
section  of  “Mental  Models”. 


Coordination  across  Agents 

The  analysis  approach  of  measuring  and  analyzing  coordination  among  the  relevant 
agents  relies  heavily  on  social  and  dynamic  network  analysis  (Figure  1)  (e.g.,  Scott, 
2000;  Carley,  2003).  The  examination  of  interaction  patterns  as  networks  within  a  C2 
environment  can  provide  information  on  a  wide  range  of  organizational  and  individual 
factors  (Wasserman  &  Faust,  1994).  The  nature  and  speed  of  information  flow  within  a 
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C2  structure  can  be  examined  through  various  network  measures  along  with  important 
structural  characteristics  of  the  C2  organization. 


r  g  h  i  j  ft 


Figure  1.  Social  network  example  illustrating  coordination  across  agents. 

The  majority  of  past  network  analysis  has  focused  on  discrete  snapshots  in  time,  which 
overlooks  the  dynamic  and  developing  longitudinal  nature  of  network  change  over  time. 
This  section  will  describe  methods  and  tools  designed  to  collect,  analyze,  and  present 
longitudinal  network  data  across  individuals,  technology,  and  the  C2  task  environment. 
Included  in  this  discussion  will  be  the  introduction  of  a  custom  observational  data 
collection  tool  for  social  and  dynamic  network  interactions  and  visualizations  that  aid  in 
the  analysis  and  presentation  of  network  changes  over  time. 

Data  Collection:  Coordination  across  agents 

The  old  computer  science  adage  of  “garbage  in,  garbage  out”  also  holds  true  for  data 
collection  and  analysis.  It  is  important  to  collect  the  appropriate  data  for  the  research 
questions  and  analyses  to  be  supported.  It  is  also  important  to  understand  many  of  the 
constraints  involved  with  data  collection  in  an  operational  environment.  For  many 
exercises  and  events,  participants  are  either  already  overloaded  with  questionnaires  or 
there  is  little  to  no  time  for  them,  not  to  mention  some  of  the  methodological  issues  with 
self-report  data.  When  computer  logs  of  interactions  and  communications  are  available 
through  collaborative  tools,  they  can  provide  a  detailed  source  of  data.  However,  face-to- 
face  interactions  are  lost  through  sole  reliance  of  such  means.  Face-to-face 
communications  can  account  for  most  if  not  all  person-to-person  interactions  in  many 
situations.  Observational  data  collection  can  capture  these  patterns  of  interactions  and  can 
be  the  primary  source  for  analysis  or  can  compliment  other  collected  data. 

In  order  to  push  the  field  of  social  and  dynamic  network  analysis  beyond  the  analysis  of 
discrete  snapshots  of  interactions  and  speculating  on  what  occurs  between  those 
snapshots,  longitudinal  data  needs  to  be  included  in  the  data  collection  plan.  Particularly 
if  we  are  interested  in  exploring  how  C2  patterns  of  interactions  occur,  evolve,  and  adapt 
over  time.  The  longitudinal  analysis  of  interactions  among  people  and  tools  requires 
timing  data  that  is  not  typically  available  through  the  traditional  means  of  data  collection 
for  social  network  analysis  (e.g.,  questionnaires  and  surveys). 

We  developed  a  custom  program  called  SNA  (Social  Network  Analysis)  Observer  to 
address  these  challenges.  SNA  Observer  was  designed  by  the  authors  and  coded  by  John 
Richardson  of  the  Computer  Information  Systems  Directorate,  Army  Research 
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Laboratory.  The  tool  is  used  to  collect  relational  data  as  to  who  is  talking  to  whom,  the 
direction  of  the  communication  flow,  and  the  duration  of  the  communication  events.  To 
facilitate  flexibility  across  multiple  hardware  and  operating  systems,  the  SNA  Observer 
was  coded  in  Java  and  has  been  tested  on  both  Microsoft  Windows  and  Macintosh 
computers.  The  tool  offers  advantages  in  flexibility,  mobility,  efficiency,  and 
interoperability  for  data  collection  focused  on  coordination  across  agents. 

Flexibility.  One  of  the  first  requirements  for  the  collection  of  longitudinal  observational 
data  of  coordination  across  agents  was  flexibility  within  the  data  collection  program. 
Face-to-face  interactions  and  small  group  formation  is  a  highly  variable  and  changing 
phenomenon  with  people  potentially  flowing  in  and  out  of  group  conversations  that 
include  cocktail  party  effects  and  constant  creation  and  dissolution  of  sub-groups  These 
characteristics  are  particularly  present  in  highly  dynamic  environments  like  many  C2 
environments.  In  addition,  individuals  are  also  interacting  with  tools  during  these  face-to- 
face  interactions,  which  is  critical  in  understanding  the  complete  distributed  cognitive 
system. 

The  SNA  Observer  allows  the  observer  to  create  multiple  groups  where  the  agents  of  the 
group  can  be  people  and/or  tools  being  interacted  with.  To  account  for  sub-group 
formation,  flexible  membership  of  individuals  in  more  than  one  group  is  supported. 
Therefore,  if  Pam,  Jim,  Kelly,  and  Dwight  are  interacting,  their  personal  icons  can  be 
grouped  together  to  represent  that  interaction  pattern,  while  a  sub-group  interaction 
pattern  between  Dwight  and  the  planning  tool  can  recorded  at  the  same  time  or  other 
interactions  by  different  agents  (Figure  2).  Furthermore,  if  additional  detail  is  needed, 
specific  communication  patterns  that  indicate  the  sender  and  recipient  can  be  illustrated 
and  recorded  (Figure  2,  session  4). 


4 


13th  ICCRTS:  C2  for  Complex  Endeavors 


Figure  2.  Display  of  the  SNA  Observer  interface.  Multiple  session  windows  allow  the 
interactions  of  multiple  group  and  sub-group  interactions  across  human  and  computer 
agents.  Directional  communication  indicated  by  Session  4  window  as  Michael  speaks  to 
the  indicated  agents. 

To  facilitate  the  quick  creation  and  modification  of  agent  interactions,  SNA  Observer  was 
designed  for  touch  screen  manipulation.  Therefore,  direct  manipulation  of  the  agents  in 
and  out  of  groups  along  with  the  detailed  interactions  among  agents  can  be  done  directly, 
quickly,  and  efficiently. 

Mobility.  Another  requirement  that  was  important  to  consider  for  the  SNA  Observer  was 
mobility.  In  order  to  support  the  observational  data  collection  over  time  in  field  settings 
and  exercises,  the  observer  must  be  able  to  go  where  the  action  and  people  are  as  they 
flow  in  and  out  of  various  patterns  and  locations.  In  order  to  enable  the  mobility  to  track 
these  interactions  and  to  collect  data  at  the  same  time,  traditional  laptop  computers  are 
difficult  to  use.  A  traditional  laptop  computer  is  designed  to  rest  on  a  tabletop  or  lap  and 
does  not  facilitate  a  standing  or  walking  position  while  being  able  to  operate  the 
computer  at  the  same  time.  A  tablet  computer  (Figure  3),  on  the  other  hand,  can  easily  be 
held  in  one  hand  and  operated  by  the  other  hand  in  a  sitting,  standing,  or  walking 
position.  A  tablet  computer  also  uses  a  touch  screen,  which  the  SNA  Observer  was 
designed  for  and  therefore  is  a  very  appropriate  hardware  solution  that  provides  the 
mobility  needed  for  observational  data  collection  over  time  for  coordination  across 
agents. 
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Figure  3.  Tablet  PC  held  by  operator  with  one  hand  with  the  freedom  to  provide  input 
with  the  other  hand. 


Efficiency.  Another  category  of  requirements  involves  efficiency  ranging  from  the 
interface  of  the  SNA  Observer  to  automatic  data  manipulation  to  facilitate  data  analysis. 
In  order  to  support  the  touch  screen  interaction  with  the  program,  a  graphical  user 
interface  (GUI)  was  designed  to  facilitate  keeping  up  with  multiple  groups  and  easy 
identification  of  agents  through  custom  icons.  There  are  default  icons  that  allow  for 
differentiation  between  individuals  and  tools  by  allowing  the  observer  to  customize  the 
icon  by  gender,  color,  general  appearance,  and  name.  It  is  also  possible  to  load  custom 
icons  such  as  photographs  of  the  observed  individuals  for  very  easy  and  clear 
identification  during  data  collection  (Figure  4). 


Figure  4.  Use  of  custom  icons  ranging  from  human  and  computer  agents  to  actual 

photographs  of  observed  agents. 

One  of  the  more  powerful  capabilities  of  the  SNA  Observer  is  its  automatic  time 
stamping  of  all  interactions  indicated  in  the  touch  screen  interface  and  the  automatic  data 
manipulation  for  data  analysis  preparation.  The  automatic  time  stamping  allows  for  the 
analysis  of  longitudinal  data  as  it  records  start  and  stop  times  along  with  the  duration 
calculations  for  each  interaction.  Therefore  the  observer  knows  who  was 
talking/interacting  with  whom,  when  it  occurred,  and  how  long  it  occurred  for.  The  time 
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note -taking  capability  also  allows  the  observer  to  record  notes  related  to  particular  events 
and  actions  as  they  occur  within  each  group. 

Interoperability.  The  other  feature  that  is  just  as  powerful  and  time  saving  is  the 
automatic  data  manipulation  the  software  does  to  prepare  it  for  analysis  with  other  SNA 
software  such  as  UCINET  (Borgatti,  Everett,  &  Freeman,  2002),  ORA  (Carley,  2003), 
and  SoNIA  (Moody,  McFarland,  &  Bender-DeMoll,  2005).  Translation  of  the  raw  data 
described  above  into  the  typical  metamatrix  format  used  in  the  analysis  of  social 
networks  can  be  extremely  time  consuming  if  not  automated.  The  automation  of  this  step 
improves  the  efficiency  of  processing  the  data  in  order  to  more  directly  feed  into  the 
network  analysis  tools  of  choice. 

Data  Visualization:  Coordination  across  agents 

The  visualization  of  the  longitudinal  data  just  described  can  greatly  improve  insight  into 
the  evolution  and  changes  in  coordination  across  agents  over  time.  Generally,  the  area  of 
social  network  analysis  uses  relatively  static  measures  of  network  change.  Data  is 
collected  in  discrete  snapshots  with  moderate  to  long  periods  between  snapshots,  which 
allows  the  researcher  to  identify  changes,  but  forces  them  to  infer  both  why  and  how 
those  changes  took  place.  The  collection,  visualization,  and  analysis  of  longitudinal  data 
eliminate  the  need  to  infer  how  interactions  and  coordination  changed  over  time.  This 
section  will  describe  a  method  and  means  of  visualizing  network  data  over  time  along 
with  the  advantages  and  disadvantages  of  this  approach. 

The  SoNIA  (social  network  image  animator)  software  (Moody,  et.  al.,  2005)  was 
designed  to  explore  dynamical  relational  data  through  the  animation  of  network 
interactions  but  not  act  as  analysis  software  (Bender-DeMoll  &  McFarland,  2006). 
Several  other  network  analysis  packages  are  available  that  cover  a  wide  range  of 
quantitative  analyses  of  networks  (e.g.,  UCINET,  ORA,  PAJEK).  The  longitudinal  data 
collected  can  be  aggregated  at  different  levels,  depending  on  the  targeted  tasks,  variables, 
or  research  questions  ranging  from  a  macro  to  micro  level  (Figure  5).  The  flexibility  to 
examine  the  network  at  these  various  levels  is  one  of  the  strengths  as  the  changes  in  the 
network  can  be  explored.  At  the  same  time,  this  flexibility  also  poses  a  challenge  to  the 
researcher  to  select  the  appropriate  level/s  of  aggregation  to  address  the  issues  at  hand. 
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Figure  5.  Interactions  represented  at  various  levels  of  aggregation  ranging  from  1  minute 
to  35  minutes  from  McFarland’s  classroom  observations  (2001).  Figure  originally 
produced  in  Bender-deMoll  &  McFarland  (2005). 

Bender-DeMoll  &  McFarland  (p.  16,  2005)  have  suggested  identifying  several  criteria 
when  creating  and  exploring  network  animations  through  SoNIA. 

1 .  What  is  the  underlying  set  of  relations  we  are  really  interested  in  looking  at, 
and  how  can  they  be  best  expressed? 

2.  What  is  the  functional  relationship  between  collected  data  and  relations  of 
interest? 

3.  What  time-scale  are  the  patterns  of  interest  likely  to  be  visible  at? 

4.  What  set  of  transformations  do  we  need  to  apply  to  get  from  the  data  to  a 
consistent  social  space? 

5.  How  might  node  and  arc  attributes  relate  to  the  pattern  of  network  structure, 
and  how  can  they  best  be  translated  into  display  variables  in  order  to  highlight  and 
explore  these  relations? 


The  visualization  of  the  network  over  time  allow  for  qualitative  analysis  of  how  and 
potentially  why  any  detected  changes  in  the  network  occurred.  Paired  with  more 
traditional  quantitative  network  measures,  the  use  of  longitudinal  network  visualizations 
allows  the  development  of  new  hypotheses,  examination  of  network  evolution  and 
adaptation  without  inference  between  discrete  snapshots  of  the  network,  exploration  of 
transition  points  of  micro  and  macro  level  network  processes,  and  analysis  of  strategic 
intervention  effects. 
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Mental  Models 

Mental  models  have  a  long  history  as  the  cognitive  representation  of  accumulated 
knowledge  and  experience  in  Psychology  and  Cognitive  Science  (e.g.,  Johnson-Laird, 
1983).  This  knowledge  in  the  head  (Norman,  1988)  represents  relationships  and  linkages 
between  domain  topics  and  concepts  and  guides  decision-making,  perception,  and 
interpretation  of  new  information.  Mental  models  obviously  have  an  important  role  and 
function  in  C2,  especially  regarding  the  establishment  and  communication  of 
commander’s  intent  between  the  commander  and  staff  (Builder,  Bankes,  &  Nordin, 

1999). 

There  is  often  a  distinction  between  types  of  knowledge  that  includes  declarative  and 
procedural  knowledge.  Declarative  knowledge  describes  awareness  or  understanding 
regarding  an  object,  event,  or  concept  (Rumelhart  &  Ortony,  1977).  Procedural 
knowledge,  on  the  other  hand,  is  an  understanding  of  “how  to”  or  the  application  of 
declarative  knowledge  in  performing  a  task  (Shank  &  Abelson,  1977).  There  is  a 
dependence  between  the  two  as  declarative  knowledge  provides  the  conceptual 
understanding  of  the  elements  to  be  used,  manipulated,  or  involved  in  procedural 
knowledge.  There  is  an  intermediate  knowledge  type  between  the  two,  however,  that 
mediates  the  translation  of  declarative  to  procedural  and  that  is  structural  knowledge. 
Knowledge  of  how  the  domain  concepts  are  interrelated  and  therefore  how  the 
declarative  knowledge  should  be  used  in  procedural  knowledge  is  comprised  in  structural 
knowledge  (Diekhoff,  1983).  Whether  structural  knowledge  is  seen  as  the  transitory  type 
of  knowledge  (Diekhoff,  1983)  between  declarative  and  procedural  knowledge  or  as  one 
of  two  dimensions  of  declarative  knowledge  (Mitchell  &  Chi,  1984),  this  type  of 
knowledge  defines  how  declarative  knowledge  is  interconnected  and  is  critical  element  in 
understanding  and  evaluating  mental  models. 

Structural  Knowledge  Measurement 

There  are  a  number  of  techniques  and  methodologies  available  to  measure  structural 
knowledge.  These  efforts  fall  into  two  required  categories  or  stages  of  knowledge 
elicitation,  1)  knowledge  elicitation  from  individual  or  population  and  2)  knowledge 
representation  and  analysis  of  collected  knowledge  (Jonassen,  Beissner,  &  Yacci,  1993). 
There  are  several  methods  within  each  stage  but  for  this  paper,  only  one  method  for  each 
stage  will  be  addressed  (for  a  review  of  others,  see  Jonassen,  et.  al.,  1993).  The  use  and 
application  of  similarity  ratings  will  be  the  elicitation  technique  discussed  while  the 
network  representation  using  Pathfinder  nets  will  be  the  knowledge  structure 
representation  technique  described. 

As  mentioned  above,  structural  knowledge  is  the  pattern  of  relationships  between 
concepts  in  declarative  memory.  These  concepts  have  varying  degrees  of  interrelatedness 
with  each  other  where  some  are  more  closely  related  to  the  targeted  concept  than  others. 
In  order  to  assess  these  relationships  and  the  varying  strengths  of  them,  individuals  can 
rate  the  similarity  between  concepts  (Jonassen  et.  al.,  1993).  Similarity  ratings  are 
typically  done  on  a  pair-wise  basis  with  a  numeric  scale  where  one  anchor  represents 
dissimilarity  and  the  opposite  numerical  anchor  represents  similarity.  These  ratings  are 
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typically  the  simplest  and  most  direct  means  of  obtaining  the  distances.  This  implies  a 
spatial  metaphor  for  depicting  the  knowledge  structure  where  the  semantic  distance  is 
congruent  with  geometric  distance  between  any  two  concepts.  In  other  words,  concepts 
like  feather  and  bird  with  a  close  relationship  would  have  a  short  distance  between  them 
compared  to  scale  and  bird  with  a  more  distant  relationship.  Past  research  has  reported 
good  reliability  in  judgments  over  time  and  high  similarity  across  experts  (Diekhoff  & 
Wigginton,  1982). 

The  representation  of  the  knowledge  structures  elicited  by  the  above  similarity  ratings 
can  be  accomplished  through  a  network  approach  using  Pathfinder  software  and 
Pathfinder  networks  (PFnets)  (Schvaneveldt,  1990).  Pathfinder  uses  the  pairwise 
proximity  estimates  for  a  set  of  concepts  and  generates  a  network  structure  (Figure  6) 
where  the  concepts  are  nodes  and  the  relations  between  concepts  are  links  in  the  network 
structure.  Closely  related  concepts  are  represented  by  their  proximity  to  one  another  in 
the  Pathfinder  knowledge  structure. 


Meta  Matrix 


Bacfr  Organization 


b,  curtir  « i;ny 


Figure  6.  Pathfinder  knowledge  structure  example  where  concepts  are  represented  as 
nodes,  relationships  as  links,  and  line  weights  represent  degree  of  relatedness. 


The  PFnets  are  constructed  using  an  algorithm  that  transforms  the  similarity  ratings  into  a 
network  structure.  The  algorithm  searches  for  the  shortest  possible  path  between  concepts 
and  maintains  those  links  between  two  concepts.  Therefore,  it  provides  all  the  links  in  the 
minimal  spanning  tree,  which  is  a  subgraph  path  linking  all  nodes  by  the  shortest  possible 
distance  (Figure  7)  (Schvaneveldt,  1990).  The  Pathfinder  algorithm  has  advantages  over 
multi-dimension  scaling  (MDS)  as  it  preserves  the  pairwaise  comparisons  better  and  it 
does  not  force  a  hierarchical  solution  like  tree  representations  do  (Jonassen  et.  al.,  1993). 
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Figure  7.  Example  of  a  minimum  spanning  tree. 


Pathfinder 

Pathfinder  has  been  in  use  for  more  than  20  years  to  represent  knowledge  structures  of 
categories  (Rubin,  1990),  scripts  (Durso  &  Coggins,  1990),  room  schemata 
(Schvaneveldt,  1990),  and  problem-solving  schemata  (Dayton,  Durso,  &  Shepard,  1990). 
The  Pathfinder  technique  has  been  used  to  identify  novices  from  experts  in  the  domains 
of  air  combat  flight  maneuvers  (Schvaneveldt,  Durso,  Goldsmith,  Breen,  Cooke,  Tucker, 
&  DeMaio,  1985),  computer  programming  (Cooke  &  Schvaneveldt,  1988),  statistical 
reasoning  and  classroom  learning  (Goldsmith  &  Johnson,  1990).  These  studies  have 
indicated  that  Pathfinder  networks  represent  knowledge  structures  in  a  meaningful  way  as 
it  identified  expert  and  novice  pilots  with  over  90%  accuracy  (Schvaneveldt,  Durso, 
Goldsmith,  et  al.,  1985)  and  accounted  for  55%  of  the  variance  in  students’  final  course 
points  (Goldsmith  &  Johnson,  1990). 

The  Pathfinder  rating  task  requires  the  participant  to  rate  the  relatedness  of  each  possible 
pairing  of  the  included  concepts.  The  relatedness  scores  are  done  on  a  1-9  scale  where  1 
is  “unrelated”  and  9  is  “related”.  Inclusion  of  15  concepts  presents  105  comparisons  to  be 
rated  and  takes  and  average  time  of  7  minutes  to  complete.  The  number  of  ratings  quickly 
escalates  as  more  concepts  are  added  (e.g.,  20  concepts  =190  comparisons). 

Pathfinder  measures  an  individual’s  knowledge  structure  or  situational  model  and  is  able 
to  quantify  several  aspects  of  individual  and  group  understanding.  The  following  list 
represents  the  different  types  of  analyses  possible  with  Pathfinder: 

•  Comparison  of  the  PFnets  across  time,  individuals,  and/or  groups.  The  similarity 
score  used  to  make  these  comparisons  is  the  proportion  of  shared  links  in  two 
PFnets  using  the  same  concepts  (theoretical  range  =  -1.0  to  +1.0)  (Interlink,  Inc., 
1996). 

o  Comparison  of  the  situational  model  to  a  referent  to  evaluate  similarity. 
Possible  referents: 

■  Subject  matter  expert  knowledge  structure 
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■  Other  groups’  knowledge  structure 

■  Model  representation 

■  Commander’s  knowledge  structure 

o  Comparison  of  PFnet  similarity  across  time  and  individuals/groups. 

■  Similarity  over  time:  change  of  the  knowledge  structure  over  time. 

■  Similiarity  between  individuals/groups:  degree  of  congruency 
among  team/group  members’  knowledge  structure  along  with 
possible  changes  over  time. 

•  Measure  of  domain  expertise  (related  to  the  concepts  included) 

o  Derived  from  Pathfinder  coherence  measure  where  coherence  is  a  measure 
of  how  consistent  were  the  participant’s  ratings.  The  coherence  score  is  a 
Pearson  Product-Moment  correlation  for  the  internal  consistency  of  an 
individual’s  ratings  (theoretical  range  =  -1.0  to  +1.0)  (Interlink,  Inc., 

1996). 

o  Past  research  indicates  a  strong  relationship  with  Pathfinder  cohesion  and 
domain  expertise  (Gaultieri,  Fowlkes,  &  Ricci,  1996;  Stout,  Salas, 

Kraiger,  1997). 

Conclusion 

The  distributed  cognitive  theoretical  framework  brings  together  various  cognitive 
attributes  typically  analyzed  only  at  the  individual  level  and  expands  them  to  multiple 
individuals,  tools,  and  the  task  environments  these  agents  are  embedded  in.  Two  of  the 
distributed  cognitive  attributes,  coordination  across  agents  and  mental  models  were 
addressed  in  this  paper  to  describe  our  efforts  of  analyzing  these  in  C2  environments 
using  existing  tools  and  techniques  and  developing  new  ones  where  needed.  The  analysis 
of  each  of  these  attributes  alone  can  be  valuable,  but  can  bring  additional  insight  and 
aspects  of  validity  when  brought  together  and  interpreted  through  a  single  theoretical 
framework. 

Each  of  the  distributed  cognitive  attributes  cover  a  wide  range  of  behavior  and  in  turn, 
possess  an  equally  wide  range  of  possible  methods  and  techniques  to  measure  them.  The 
longitudinal  network  methods  addressing  coordination  across  agents  and  the  structural 
knowledge  methods  targeting  mental  models  are  only  two  tools  for  the  C2  researcher  to 
place  in  their  toolbox.  They  are  powerful  methods  and  tools  that  can  be  used  in  a  wide 
range  of  situations  and  research  questions  but  they  are  not  the  only  way  to  tackle 
distributed  cognitive  issues  for  these  attributes.  Additional  efforts  in  this  area  to  further 
explore  and  refine  these  and  other  methods  is  needed  if  continued  rigor  and  insight  is  to 
be  gained  of  C2  as  it  is  examined  through  the  distributed  cognitive  lens. 
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